Environment classification plays an important role in facilitating the best sound processing for a hearing impaired user of a hearing device. A large amount of processing power is today used to determine the environment based on acoustical sound input. This process has is limits in determining the environment as it needs a constant varying sound in order to adapt. In addition to the environment classifier in the hearing device, the hearing device user has the possibility to change the pre-set program in the hearing device to optimize the sound processing in a given situation or environment. This involves user intervention which not all users can do or want to do due to physical limitations such as other impairment or stigmatization. Therefore it is desirable to provide a system that is configured to optimize the sound processing based on more than the sound input, and without user intervention.